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Abstract 

We set up a signal-driven scheme of the chaotic neural network with 
the coupling constants corresponding to certain information, and investi- 
gate the stochastic resonance-like effects under its deterministic dynamics, 
comparing with the conventional case of Hopfield network with stochas- 
tic noise. It is shown that the chaotic neural network can enhance weak 
subthreshold signals and have higher coherence abilities between stimulus 
and response than those attained by the conventional stochastic model. 
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1 Introduction 

Stochastic resonance(SR) is known as a phenomenon in which the presence of 
noise helps a nonlinear system in amplifying a weak (under barrier) signal Q. 
The features of systems exhibiting SR seem to be applicable to some natural 
systems such as sensory neurons, which are noisy and operate as threshold sys- 
tems. Since SR produces an information-transmitting phenomenon, its positive 
role in the neuronal processes is to be revealed. In fact, a single neuron model 
described by the FitzHugh-Nagumo (FHN) equations exhibits SR behavior 
and this SR effect is found in the real sensory neurons located in the tail fan of 
crayfish Q|. The FHN equations driven by white noise and an arbitrary aperi- 
odic signal are also examined in the context of excitable systems with threshold 
dynamics [flj |) . One can expect that the SR effect will be more pronounced in 
an ensemble of systems than in a single system. In view of a collective response 
of globally coupled bistable systems to periodic forcing, a neural network with 
dynamics of the Hopfield type || is studied, under the assumption that white 
noise and the periodic signal are identical for all neurons . 

Given the basic three ingredients, that is, a form of threshold, a source of 
noise and a weak input signal, SR can generally be observed in a large variety of 
systems. Considering the fact that deterministic chaos resembles the feature of 
noise and provides a source of fluctuation, we have a natural question whether 
SR-likc behavior can be observed in deterministic dynamical systems in the 
absence of noise. Two different approaches to this problem are known. One way 
is to substitute the stochastic noise by a chaotic source. This situation in which 
the chaos is supplied as an additive noise, resembles the conventional setup for 
SR and yields SR-like enhancement as expected || g|. The other approach is 
to use the intrinsic chaotic dynamics of a nonlinear map. No external source is 
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necessary to provide the randomness. This method generates a sort of activated 
hopping process which is then synchronized by a weak periodic signal J[o| pd| . 

Until now, diverse types of chaos have been confirmed at several hierar- 
chical levels in the real neural systems from single cells to cortical networks 
(e.g. ionic channels, spike trains from cells, EEG) |Q. By producing chaos 
as effective noise spontaneously, biological systems may enhance their functions 
through signal amplification. This scenario is likely to have occurred even in 
the associative memory dynamics. The chaotic neural network model []l3[ is 
known as a framework beyond the Hopfield neural network A with only equi- 
librium point attractors. Its dynamic retrieving and learning features have 
been studied [[u], [H], In this paper, we set up a signal-driven scheme of the 
chaotic neural network with the coupling constants corresponding to certain in- 
formation, and investigate the SR-like effects under its deterministic dynamics, 
comparing with the conventional case of the Hopfield network with stochastic 
noise. 

2 Models 

For N neurons connected by synaptic couplings Wij with wu = 0, the system 
(signal-driven scheme) is described by 

Xi(t + l) = f(hi(t)+Si(t)) , (1) 

where Xi. output of neuron i{— 1 < Xi < 1), : synaptic weight from neuron j 
to neuron i, /: output function defined by f(y) = tanh(y/2e) with the steepness 
parameter e, hf. internal potential, S^. contribution of external input signal. 
This form is a simple and possible incorporation of stimuli as the changes of 
neuronal activity. In the case of chaotic neural network (CNN) the internal 
potential is given by 

hi(t) = m(t) + Q(t) , (2) 

N t 

j=l d=0 
t 

Ci(t) = -a^fc^i-d)-^ , (4) 

d=0 

where B. L : threshold of neuron i,kf(k r ): decay factor for the feedback (refrac- 
toriness) (0 < fc/, k r < 1), a: refractory scaling parameter. Owing to the 
exponentially decaying form of the past influence, the dynamics of {rji} and 
{Ci} can be described as follows : 

N 

Vt (t) - k fm (t-l)+J2^j(t) , (5) 

Q(t) = k r Q(t - 1) - aX^t) - 6,(1 - k r ) . (6) 

When a = kr = k r = 0, the network corresponds to the conventional discrete- 
time Hopfield network (we call the Hopfield network point (HNP)): 

N 

X i (t + l)=f(j2™ijXj(t)-0i) ■ (7) 
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We also look into the case that stochastic fluctuations are attached to HNP in 
Eq.(7): 



N 

hi(t) = WijXj (t) -9 t + E (t) , (8) 
i=i 

where (t) is a neuron- independent Gaussian white noise defined by < (t) >= 
and < Fi(t)Fj(t') >= D 2 S tt /5ij, and D is the noise intensity parameter. We 
call Eq.(8) a stochastic neural network(SNN). 

The synaptic configuration {uiij} is determined by storing pattern informa- 
tion in the network as minima of the computational energy : 



E -- 
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l^w^XiXj (9) 



at HNP. This is done by using a local iterative learning rule |L7J for p patterns 
{$} = (tf , • • • , {n = 1, • • • ,p; = +1 or - 1) in the following form : 



w new = old 

l 3 1 3 



EK ( 10 ) 



with 



5^ = 1=9(1- (11) 



N 

where 7^ = £f J^fci w ij£j anc ^ ^C 1 ) * s the un ^ s ^ e P function 



3 Simulations and Results 

To carry out computational experiments, we consider a network with N = 156, 
{9i\ = and e — 0.015(unless otherwise stated), and use non-orthogonal 20 
random patterns R 1 ~ R 20 as a set of external signal: {Si} — s{£f = 
1, • • ■ , 20). s is the strength factor of signal. {£f } (i = 1, • • ■ , iV) is represented 
by 12 x 13 binary data and a half of £f s has +1 and the other half has —1. 
10 patterns i? 1 ~ R 10 are stored with the above learning rule of Eq.(10) (p — 
10), and then the corresponding multi-stable landscape is made on the network 
dynamics. 

As the temporal input signal {Si(t)}, we take two kinds of signals. One is a 
random train composed of stored patterns R 1 ~ R 10 (e.g. [i? 3 — > R 7 — > R 2 — > 
• • •]) with a duration Xj for every pattern and the other is a similar sequence of 
non-stored patterns R 11 ~ i? 20 . Figure 1 shows typical examples of the results 
of temporal behaviors of the neural networks to input signal {5, (t)}. An input 
random train of stored patterns (Tj = 100) is given in Fig. 1(a) by the relative 
overlap with pattern i? 4 : 

1 N 

A? E 5 <(*)£ ' & = ■ ( 12 ) 
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Figure 1: (a) Time series of the input signal {Si(t)} with Tj = 100 used in our 
simulations, (b) The behaviors of the output response {Xi(t)} in CNN with 
different values of the refractory parameter a for the signal (s=0.5). (c) Those 
in SNN with different values of the noise intensity parameter D. In (a) ~ (c) 
the ordinate is the relative overlaps with the pattern R 4 [Eqs.(12),(13)]. 



4 



Responses of CNN (kf = 0.1, k r — 0.7) and SNN to this signal with strength 
s = 0.5 are shown in Fig. 1(b) and (c) by 

1 N 

In CNN [Fig. 1(b)], as we can see from three cases where a is 0.125, 0.375 and 
0.625, the performance of output response is largely affected by its refractory 
scale parameter a. The response fits well the input signal when a is 0.375. From 
Fig. 1(c), similar behaviors are observed for SNN against the noise intensity 
parameter D{= 0.25,0.75 and 1.5), except for the appearance of noise-driven 
fluctuations. 

To evaluate the coherence between the signal {Si(t)} and the response {Xi(t)}, 
we introduce the correlation coefficient r and the discrimination efficiency n. 
These quantities are defined as 

r = Dev(mj)Dev(m%) 

[Dev(mj) 2 ] V2 [Dev(m$ 2 ] V 2 

and 



n=-^2Si(t)Xi(t) , Si = Si/ a , (15) 
»=i 

where Dev(Y) is the deviation Y — Y and the overbar denotes an average over 
time, n naively means the rate of information transfer from stimulus to response. 

The numerical results of r and n plotted against refractory factor a are 
given in Fig. 2, calculated using the temporal data at every different a values. 
The results for {£,(£)} of stored patterns R 1 ~ R 10 are in contrast with the 
results for {<%(*)} of non-stored patterns R 11 ~ R 20 in the range of a larger 
than about 0.2 wherein the response is hopping out and into a well (minimum 
of the energy) corresponding to a stored pattern state. This movability causes 
an increased coherence with stored pattern stimuli. Under the same conditions 
of input signal, dependences of r and n on the noise intensity D in SNN are 
examined as shown in Fig. 3. Between stored and non-stored pattern signals, a 
quite difference like in CNN is also confirmed. Comparing the results for stored 
pattern signal, we can see that in CNN there appears a projected plateau for 
both r and n, and in this flat region both of their values are kept very close 
to 1. Contrary to this, in SNN r and n have gradual variations with D and 
n degrades faster than r, which seems to be consistent with conventional SR. 
This comparison indicates that CNN can cause SR-like phenomena with high 
performance which cannot be attained in SNN. 

In CNN, the stronger the signal strength s is, this a-range of the flat plateau 
becomes wider. Conversely, this range becomes narrower for weaker s. We illus- 
trate this effect as coherent a-range versus signal strength s in Fig. 4, together 
with the relationship of maximum Lyapunov exponent Ai jls[ ] to a when there 
exists no signal forcing. As s decreases, the a-range (r > 0.9) becomes narrow 
and at last disappears. Then the a value coincides with a* at the sudden rise 
point of the Lyapunov exponent Ai. This fact tells that sensitive and flexible 
responses happen around the boundary between order and disorder, in other 
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Figure 2: Correlation coefficient r (solid line) and discrimination efficiency n 
(dotted line) as a function of refractory parameter a in CNN simulations with 
Eq.(2). The results for the input signals of stored and non-stored pattern trains 
are shown. 
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Figure 3: r (solid line) and n (dotted line) as a function of noise intensity 
parameter D in SNN simulations with Eq.(8). 
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Figure 4: Dependence of the coherent a-range (r > 0.9) and the r max point 
on the input signal strength s (the left-axis) in CNN, shown together with the 
maximum Lyapunov exponent Ai (the right-axis) as a function of parameter a 
in CNN with no signal forcing(s = 0). 

words, the edge of chaos |lj|. At a < a*, the refractory term (— aXi) in Eq.(6) 
makes a well (minimum of the energy) shallow in effect and helps the input sig- 
nal drive the network state. On the other hand, at a > a* the chaotic attractor 
is driven by the input signal and stabilized to the corresponding network state. 

Figure 5 shows return maps of the internal potential hi for a neuron (i = 12) 
in CNN with no signal forcing (s = 0) when a > a*. In the case of a — 58/128, 
the trajectories of hi are attracted into the region of square (—0.5 ~ 0.5) such 
that controllable by the input signal s = 0.5. When a = 96/128 (a deeper 
chaotic state), however, the trajectories are attracted outward so that chaos 
cannot be suppressed by the input signal s — 0.5. 

We have investigated all the above phenomena in other conditions of the 
parameters and the input signals, and have found similar results and overall 
tendency. 

4 Conclusion 

We have shown that the chaotic neural network can enhance weak subthresh- 
old signals and have higher coherence abilities between stimulus and response 
than those attained by the conventional stochastic neural network model. The 
high coherent response is found to arise around the edge of chaos. This implies 
that some of SR phenomena may be realized by the inherent properties of de- 
terministic nonlinear systems without any external noise. Analytical study to 
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Figure 5: Return maps of the internal potential for the 12th neuron (i = 12) in 
CNN with no signal forcing in the cases of a = 58/128 and 96/128 (> a*). 



explain these results will be important in our future work. The coherent re- 
sponse concept is expected to be related to researches in cognitive neuroscience 
from dynamical system viewpoints [po| 
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